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Abstract 


Data  Envelopment  Analysis  (DEA),  a  new  methodology  based  on 
mathematical  programming  models,  provides  an  approach  to  evaluation  of 
the  relative  efficiency  of  organizations,  especially  not-for-profit 
organizations  which  have  multiple  outputs  and  inputs.  This  paper  uses 
an  artificial  data  base  to  evaluate  DEA  relative  to  other  alternatives 
such  as  ratio  and  regression  analyses.  The  results  of  this  study 
generally  favor  DEA  not  only  for  identifying  inefficiencies  but  also  for 
locating  their  sources  and  estimating  their  amounts  in  particular 
DMUs  (Decision  Making  Units).  Statistical  regressions  performed  very 
poorly, e  pgr  se,  as  well  as  by  comparison.  Reasons  for  the  poor 
performance'foT  these  customary  statistical  regression  approaches  are 
indicated  along  with  possible  ways  of  improving  this  performance. 
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1 .  Introduction 

Data  Envelopment  Analysis  (DEA)  is  a  new  efficiency  measurement 
methodology  developed  by  A.  Charnes,  W.  W.  Cooper,  and  E.  Rhodes  as  set 
forth  in  [12]  [l 3]  and  [14]v^  It  is  designed  to  measure  the  relative 
efficiency  of  Decision  Making  Units  (DMUs)  which  use  multiple  inputs  to 
produce  multiple  outputs  even  when  the  underlying  production  function  is 
not  known  and  where,  additionally,  these  functions  may  also  be  multiple 
in  character.  This  contrasts  with  the  situation  for  statistical 
techniques  and  theory,  e.  g.,  as  employed  in  economics,  where  either  the 
underlying  production  function  must  be  known,  or  at  least  its  parametric 
form  must  be  assumed  before  it  can  be  used  to  evaluate  efficiencies  and 
where,  usually,  a  single  functional  form  is  also  assumed.  See,  e.  g., 

Feldstein  [18].  See  also  [32]  and  [33].  The  latter,  regression  approaches, 
are  thus  limited,  especially  in  the  case  of  public  sector  institutions  such 
as  hospitals,  etc.,  where  programs  and  activities  are  even  less  readily 
identified  for  such  assumptions  than  is  the  case  in  industrial  production. 

DEA  has  now  been  applied  to  several  types  of  organizations  including 
education  [5]  [6],  health  care  [4]  [29],  Navy  recruiting  [22],  and  criminal 
court  systems  [21].  Nevertheless  something  more  is  required  and,  in 
particular,  the  validity  and  reliability  of  DEA  in  locating  inefficient 
DMUs,  identifying  the  inputs  (and/or  outputs)  where  the  inefficiencies 
occur  and  estimating  their  amounts  or  magnitudes  all  need  to  be  evaluated. 

One  way  to  approach  this  task  is  via  a  situation  in  which  the  identity  of 
the  truly  inefficient  units  is  known  along  with  the  sources  and  amoints  of 
this  inefficiency.  This  paper  therefore  attempts  to  evaluate  DEA  through  use 
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of  an  artificial  data  base  where  the  efficient  and  inefficient  DMUs  are  all 
known  in  numerical  detail.  DEA's  performance  is  then  compared  with  other 
commonly  employed  techniques  such  as  ratio  and  regression  analyses. 

Regression  and  ratio  analyses  were  selected  for  these  evaluations 

because  they  are  widely  used  in  fields  like  health  services,  which  is  the 

field  we  shall  use  to  guide  our  data  base  construction.  In  this  paper 

we  restrict  our  examination  only  to  some  of  the  fairly  simple  forms  of  ratio 

and/or  regression  approaches  that  are  in  wide  use.— ^  More  sophisticated 

regression  techniques  such  as  the  translog  function  and  other  so-called'flexible 

functional  form"  approaches  are  considered  elsewhere.  See  Sherman  [29]. 

The  following  section  describes  how  the  data  base  was  constructed  and 

section  3  discusses  the  data  base  that  was  developed.  Section  4  describes 
the  version  of  DEA  that  will  be  used  while  sections  5,  6  and  7  discuss  the 
results  of  applying  OEA,  ratio  and  regression  analyses  to  this  data  base.  The 
resulting  comparisons  are  summarized  in  section  7  with  respect  to  the 
ability  of  these  techniques  to  identify  and  distinguish  between  efficient 
and  inefficient  DMUs.  Section  8  then  extends  the  uses  of  DEA  to  locating 
and  estimating  the  amounts  of  inefficiencies  in  particular  DMUs  in  ways 
that  are  not  generally  available  when  the  ratio  or  regression  approaches 
are  used.  A  concluding  section  then  discusses  some  of  the  shortcomings  found 
in  these  other  approaches  and  indicates  where  they  differ  from  DEA  and 
how  some  of  their  shortcomings  might  be  repaired. 

1/  Similarly, only  one  version  of  DEA  is  used  and  no  attempt  is  made  to 
distinguish  between  various  types  of  efficiencies  such  as  scale  vs. 
technical  efficiencies  and  other  sources  of  inefficiency  such  as  are 
examined  in  [3].  Finally,  we  did  not  use  statistical  principles  of 
experimental  design,  such  as  random zation  replication,  etc.  to  develop 
our  data  base,  as  was  done  in  [2],  and  hence  can  make  only  limited  use  of 
statistical  significance  tests  and  like  devices  for  generalizing  our  results. 
Our  purpose  is  rather  to  supply  insight  of  potential  value  on  the  use  of  the 
techniques  we  study  rather  than  to  secure  generalizations  for  the  different 
data  situations  that  might  be  encountered  in  actual  practice.  See  conclud¬ 
ing  section  of  this  paper.  See  also  [2]  where  similar  conclusions  are 
reached  in  experiments  conducted  in  accordance  with  the  usual  principles  of 
the  statistical  design  of  experiments. 
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2.  Model  Structure  and  Data  Generation 

The  artificial  data  set  was  constructed  by  defining  a  hypothetically 
"known"  technology  which  applies  to  all  Decision  Making  Units  (DMUs)  and 
defines  efficient  input-output  relationships  for  each  of  them.-^ 
Inefficiencies  which  were  explicitly  introduced  for  certain  DMUs  take  the 
form  of  excess  inputs  used  for  the  output  levels  attained.  Hence,  a  DMU 
that  achieves  its  output  level  by  using  only  the  amount  of  inputs  required 
by  this  hypothetical  technology  is  efficient  while  a  DMU  that  uses  more 
than  the  required  amount  of  any  input  is  inefficient.  To  make  the  inputs 
and  outputs  easier  to  recognize,  they  are  referred  to  and  labelled  in  the 
context  of  a  hospital  study  as  one  area  of  potential  interest.  See 
Sherman  [29].  We  assume  that  these  hospitals  are  all  public  (not-for- 
profit)  institutions  so  that  the  usual  profit  calculus  and/or  price-weighted 
reductions  to  a  scalar  measure  of  efficiency  evaluation  are  not  wholly 
appropriate. 


1/  Knowledge  gained  from  the  study  of  Massachusetts  hospitals  reported  in 
•  [29 J  was  used  In  the  cnolce  or  inputs  and  outputs  and  in  the  construction 

of  the  data  set. 
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The  set  of  artificial  hospital  data  generated  for  our  simulation  consisted 
of  three  outputs  produced  with  three  inputs  during  a  one  year  period  of  time^/ 
as  follows: 


Outputs 

Input  8 

Regular  patient*  care/year 
(patients  treated  in  one  year 
with  average  level  of  inputs 
for  treatment) 

xl: 

Staff  utilized  in  terms 
of  full-time  equivalents, 
i.e.,  (FTE  s)/year 

Severe  patient*  care/year 

(patients  treated  in  one  year 
with  severe  illness  requiring 
higher  input  levels  than 
regular  patients  for  more 
complex  treatment). 

x2: 

Number  of  hospital  bed 
days  available/year 

Teaching  of  residents 
and  intems/year 
(number  of  individuals 
receiving  one  year  of  training) 

x3 : 

Supplies  in  terms  of 
dollar  cost/year 

♦measured  in  terms  of  number  of  patients  treated 
The  data  set  to  be  generated  was  for  15  hypothetical  hospitals  which  we 
label  as  HI,  H2,  . ..,  HI 5,  to  represent  the  pertinent  DMUs.-^  They  are  all 


assumed  to  achieve  their  outputs  via  a  conmon  production  process,  which 
they  may  use  efficiently  or  inefficiently.  The  resulting  observed  values 
are  then  constructed  in  a  manner  that  we  shall  shortly  describe. 

In  this  study  we  shall  focus  on  input  inefficiencies,  by  which  we 
mean  that  one  or  more  of  the  above  inputs  may  be  used  in  excess  to  obtain 
a  particular  hospital's  output  values.  Although  we  could  also  similarly 
study  output  deficiencies  (in  the  form  of  output  shortfalls  from  given 
inputs)^e  shall  not  lengthen  the  paper  to  undertake  that  study  here. 


In  any  case  the  known  values  of  the  per  unit  inputs  for  efficient  pro¬ 
duction  are  given  in  Exhibit  1  inserted  at  the  end  of  this  paper. 

T7TT  e.,  we  are  considering  all  data  as  annual  rates. 

^/Subdivisions  may  also  be  used  such  as,  e.  g.,  the  suroical  units  within  erch 
hospital  that  were  studied  in  [29  ]. 

3/An  output  shortfall  approach  from  given  inputs  is  used  in  [2]. 
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The  usual  regression  approach  to  efficiency  and  related  types  of 
economic  analyses  in  multiple  output  situations  uses  a  single  aggregate 
function  of  a  linear  or  logarithmic  variety  in  which  total  cost  is 
regressed  against  the  observed  output  values.  See,  e.  g.,  [18].  This 
approach  carries  with  it  a  variety  of  assumptions--/  which  we  shall  try  to 
favor  in  our  construction  by  using  the  same  prices  and  a  common  technology 
for  all  DMUs.  We  shall  not  assume  that  all  DMUs  operate  on  their  efficiency 
frontiers,  however,  but  we  shall  otherwise  proceed  in  accordance  with  the 
usual  methods  of  estimation,  testing  and  analyses  that  have  been  commonly 
employed  in  regression  studies  of  health  services  and  related  fields. 

To  make  the  sense  of  this  discussion  more  precise,  we  present  our 
expressions  for  generating  the  inputs  required  for  efficient  operations 
by  any  hospital  in  the  following  form: 


3 

xij  =  airj  ^rj 

■J  y.-'J 

where 

x..  =  amount  of  input  i  used  per  year  by  hospital  j 

■  J 

y  .  =  amount  of  output  r  produced  per  year  by  hospital  j 

■  J 

airj  =  aR10unt  inPut  i  used  per  unit  of  output  r  by 
hospital  j  during  the  year. 


1/  See,  e.  g.,  Sato  [27 ‘ 


2/  A  use  of  DEA  to  distinguish  coefficients  for  input-output  analyses 
derived  from  data  for  efficient  and  inefficient  sets  of  operations  nay 
be  found  in  Schinnar  [28]. 
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These  a^rj-  values,  which  are  fixed  constants,  represent  an  efficient  set 
of  coefficients  which  may  be  used  to  generate  the  inputs  required  for  any 
observed  (or  planned)  level  of  outputs.  In  some  cases  we  will  assign 

A 

values  a.^  >  a,.^  for  some  i,  r  and  j  to  represent  managerial  hospital) 
inefficiencies  which  yield  values 
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Z  a 
r-1 


irj 


(2) 


with  x^j  >  Xjj  when  inefficiencies  are  present. 

The  efficient  air  values  are  given,  free  of  any  of  the  j  “  1,...,  15 
hospital  identification  subscripts,  in  Exhibit  1.  These  values  are  the  same 
for  all  hospitals  so  that  a-^  “  .004  FTE/patient  represents  the  efficient 
labor  requirement  in  Full  Time  Equivalent  units  per  regular  patient. 
Similarly  a^  =  *005  FTE/patient  represents  the  efficient  requirement  for  a 
severe  patient  and  *  .03  FTE/training  unit  represents  the  efficient 
requirement  to  train  one  new  resident /intern  during  a  year. 


Analogous  remarks  apply  to  the  values  a21  7  bed  days/patient,  and 

a22  =  9  bed  days/patient  for  regular  and  severe  patients,  respectively, 
shown  in  the  Bed  Days  column  of  Exhibit  1.  The  blank  shown  in  the  row 
for  Training  Units  in  this  column  means  that  a^  =  0  applies.  That  is, 
no  Bed  Days  enter  into  the  training  outputs. 

Finally,  a^  =  $20/patient  and  a^2  *  $30/patient  represent  the 
efficient  level  of  supplies  required  per  regular  and  severe  patients, 
respectively,  while  a^  =  $500/training  unit  is  the  coefficient  for 
efficient  training  operations  in  output  r  =  3.  Putting  this  i  =  3  input 
in  dollar  units  avoids  the  detail  that  would  otherwise  be  needed  to 
identify  the  different  types  of  supplies  that  would  be  required  for 
teaching  and  for  different  types  of  patient  treatments. 
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DEA  does  not  require  reductions  to  cost  equivalents.  The  various 

outputs  and  inputs  may  be  specified  in  different  units  of  measure  and, 

indeed,  it  can  be  shown  that  the  resulting  DEA  efficiency  value  is 

independent  of  the  units  of  measure  used  in  any  output  or  input. On 
the  other  hand  reductions  like  these  are  required  for  the  ratio  and 

regression  measures  we  shall  also  study.  Therefore  we  next  show  how  the 

efficient  costs  are  derived  to  obtain  this  part  of  our  data  set.  This 

is  done  via  expressions  of  the  form, 

3 

Cr  =  .?  ki  air  r  =  1,  2,  3,  (3) 

where  we  have  omitted  the  index  j  for  hospital  identification  because  only 
efficient  costs  are  being  considered.  Here  represents  the  cost  of  the 

ith  input  requirement  for  the  rth  output  under  efficient  operations  where 
k]  =  $10,000/FTE 

k2  =  $1 0/bed  day  (4) 

k3  =  $1 /supply  unit. 

These  data  are  then  combined  with  the  preceding  a.,  values  to 
obtain 

c<|  =  k-j a^ i  +  k2a21  +  k3a31  =  $130/regular  patient 

c2  =  kla12  +  k2a22  +  k3a32  =  $170/severe  patient  (5) 

c3  =  k]a]3  +  k2a23  +  k3a33  =  $500/training  unit. 

These  are  the  formulas  used  at  the  bottom  of  Exhibit  1  to  produce 

the  efficient  cost  of  outputs  shown  in  the  last  column  In  the  body  of  the 
table. 

y  Provided,  of  course,  that  these  same  units  of  measure  are  used  for  the 
specified  output  (or  input)  in  the  data  for  every  DMU.  See  Charnes, 
Cooper  aid  Rhodes [ill  and  [12].  See  also  Rhodes  [25]. 
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3.  Data  Base  Development 

We  now  turn  to  Exhibit  2  which  reflects  the  composition  of 
inefficient  and  efficient  hospitals  included  in  our  data  base.  The 
hypothesized  "actual"  (or  observed)  inputs  per  unit  output  used  by  each 
hospital,  whether  efficient  or  not,  are  listed  in  Exhibit  2,  columns  9-16 
with  inefficient  input  levels  per  unit  of  output  denoted  by  CD- 
Column  17  reflects  the  actual  vacancy  rate  (%  of  unused  bed  days  available 
during  the  year)  where,  as  noted  in  Exhibit  1,  an  efficient  hospital  is 
expected  to  have  a  5 %  vacancy  rate. 

We  develop  the  actual  inputs  used  for  each  hospital  in  the  manner  we 
have  already  described  by  first  selecting  an  arbitrary  set  of  output  values 
for  each  of  the  hospitals  listed  in  the  left-hand  stub.-^ Teaching  units 
per  year  are  reflected  in  column  6,  regular  patients  treated  during  the 
year  are  in  column  7,  and  severe  patients  treated  during  the  year  are  in 
column  8. 

Other  ways  of  summarizing  patient  care  outputs  for  later  use  are 
included  in  columns  4  and  5.  Column  4  reflects  total  patients  as 
the  sum  of  column  7  and  column  8.  Column  5  reflects  the  percentage  (%) 
of  severe  patients  treated  which  is  based  on  (column  8)  +■  (column  4)  X  (100). 

We  develop  this  percentage  output  measure  because  it  reflects  output  data 

p  / 

in  a  form  which  is  often  used  to  evaluate  efficiency  in  many  real  data  sets.- 
The  inputs  used  by  each  hospital  to  produce  the  outputs  in  columns  6,  7, 
and  8  are  reflected  in  columns  1,  2,  and  3.  Column  1  contains  the  full  time 
equivalents  (FTE  s)  of  labor  years  used.  Column  2  has  the  bed  days/year 

which  were  available  and  column  3  gives  the  supply  dollars  used  during  the  year. 

]_/  Although  these  values  "could  have  been  selected  by  statistical  principles— 
e.g.,  of  an  experimental  design  variety—there  seemed  to  be  little  point  in 
doing  so  because  our  objective  was  to  secure  insight  rather  than  the  kinds  of 
general i zabi 1 ity  that  require  statistical  tests  of  significance.  See  [2], 
however,  for  a  study  of  the  latter  type. 

ZJ  See  the  discussion  in  Sherman  [29]. 
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The  values  in  columns  1,  2,  3  reflect  mixtures  of  efficient  and 
inefficient  utilization  of  resources  because  of  the  way  they  were  derived. 

We  can  clarify  this  by  means  of  Exhibit  3  which  illustrates  how  the  data 
for  HI,  an  efficient  DMU,  and  H15,  an  inefficient  DMU,  were  constructed. 

HI  is  efficient  and  therefore  used  the  same  inputs  per  unit  outputs  as 
the  structural  model  in  Exhibit  1.  During  the  year,  HI  provided  care  for 
3000  regular  patients,  2000  severe  patients,  and  50  training  units  of 
service.  It  therefore  utilized  (.004)(3000)  +  (.005)(2000)  +  ( . 03 ) ( 50 )  =  23.5 
FTE s  in  that  year.  HI 5  produced  the  same  outputs  as  HI  but  was  inefficient 
in  its  use  of  certain  inputs.  It  used  .005  FTEs  /regular  patient,  while  it 
adhered  to  the  structural  model  FTE  usage  rates  for  severe  patients 
(.005  FTEs  /patient)  and  training  (.03  FTEs  /training  unit).  Hi 5  therefore 
used  ( .005) (3000)  +  (.005)(2000)  +  (.03)(05)  =  26.5  FTEs  /year  to  produce 
the  same  outputs.  Similarly,  HI 5  is  inefficient  in  the  number  of  bed  days 
used  and  supply  dollars  used  per  regular  patient  but  is  efficient  in  the 
amount  of  bed  days  and  supply  dollars  consumed  for  severe  patients  and  for 
supply  dollars  used  for  teaching  outputs.  Bed  days  and  FTEs  and  supply 
dollar  inputs  are  also  calculated  in  Exhibit  3  to  further  illustrate  the 
way  the  data  base  was  constructed. 

The  number  of  FTEs  ,  bed-days,  and  supply  dollars  inputs  were  calculated 
as  illustrated  in  Exhibit  3  for  each  hospital  based  on  the  arbitrarily 
assigned  output  mix  of  regular  patients,  severe  patients  and  training  units 
and  actual  efficient  or  inefficient  input  per  unit  output  rate  reflected 
in  Exhibit  2. 


Certain  relationships  posited  in  the  structural  model  are  generally 


not  known,  like  the  actual  amount  of  staff  time  and  supplies  that  are 
required  to  support  each  intern  or  resident  at  a  hospital.  We  nevertheless 
explicitly  introduce  these  relationships  to  determine  if  the  efficiency 
measurement  techniques  we  will  apply  can  uncover  them.  Before  proceeding, 
however,  it  should  perhaps  be  noted  that  when  the  underlying 
structural  model  is  known,  the  determination  of  which  DMU s  are  inefficient 
can  be  directly  determined  and  techniques  such  as  we  will  be  considering 
would  be  unnecessary  for  purposes  of  efficiency  evaluation. 

4.  The  PEA  Model : 

The  Charnes  Cooper  Rhodes  (CCR)  model  for  data  envelopment  analysis 
which  we  will  use  assumes  the  following  form: 


Objective: 


max  h„  = 


r=l  Ur  yro 


m 

E  w.  x. 
i=i  1  10 


Constraints: 


Less  than 
Unity 

Constraints 


E  u  y  • 
>  r=l  r  rJ 
m 

E  w.  x.. 
i=l  1  1J 


;  j  =  1 . 15 


(6) 


Positivity  .  0  <  uf  ;  r  =  l,...,s 

Constraints1  0  <  w^  ;  i  =  l,...,m 

Data: 

Outputs:  yrj  =  observed  amount  of  rth  output  for  jth  hospital 

Inputs:  x^j  =  observed  amount  of  it*1  input  for  j^  hospital. 

1/  Other  models  which  might  have  been  used  can  be  found  in  [ 3  ]  and  [  15 ]. 
See  also  [  16  ] . 


This  model  is  therefore  in  fractional  programming  form  with  fractional 

constraints.  As  noted  in  Charnes,  Cooper  and  Rhodes  [13  ]  it  may  be 

replaced  by  an  ordinary  linear  programming  model  that  also  has 

non-Archimedean  conditions  imposed  on  the  variables  for  what  are  here 

referred  tQ  as  positivity  constraints.^ 

We  shall  not  enter  into  this  kind  of  development  but  shall  instead 

try  to  explicate  what  is  happening  in  our  DEA  analysis  by  means  of  the 

above  model.  First  we  observe  that  the  efficiency  ratings  are  all 

restricted  to  an  upper  limit  of  unity.  One  of  the  j  =  1,...,15  hospitals, 

when  singled  out  for  efficiency  evaluation, is  represented  in  the  objective 

as  well  as  the  constraints.  By  virtue  of  the  latter  condition  we  must 

have  max  ii  =  h  *  <  1.  Furthermore  all  observations  y..  and  x..  are 
o  o  -  Jij  ij 

positive  so  that, together  with  the  positivity  imposed  on  the  variables, 
we  will  also  have  0  <  hQ*  <  1  with  hQ*  =  1  when  and  only  when  DMU0,  the 
DMU  being  evaluated,  is  efficient. 

Qualifications  need  to  be  entered  to  allow  for  the  presence  of  slack 

2/ 

in  the  corresponding  linear  programming  model.-  We  will  not  treat  this  topic 

in  rigorous  detail  in  the  present  paper  but  will  instead  supply  an  illustration 

with  accompanying  discussion  that  will  provide  insight  into  what  is 

involved.  Here  we  need  only  say  that  when  slack  is  present  in  some  input 

then,  with  efficiency,  that  input  may  be  reduced  to  a  new  input  level  by 

TJ  See  Charnes,  Cooper,  Lewin,  Morey  and  Rousseau  [10  ]  for  a  precise 
development. 

2/  Any  slack  which  occurs  in  (6)  is  simply  the  complement  of  an  efficiency 
rating  but  the  development  in  [10  ]  provides  a  way  of  identifying  the 
presence  of  non-Archimedean  values  in  (6)  with  slack  in  the  corresponding 
linear  programming  model. 
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removing  the  slack  without  affecting  any  output  or  any  other  input.  Hence 
the  input  which  involved  this  slack  was  excessive  and  the  operation  could 
not  have  been  efficient. 

Bearing  this  in  mind  we  next  initiate  our  DEA  analysis  by  reference 
to  the  data  of  Exhibit  2  after  which  we  shall  attempt  to  compare  the 
resulting  efficiency  ratings  with  cost  ratio  and  regression  approaches 
applied  to  this  same  data  base. 

5.  Applications  to  Artificial  Data  Base. 

Applying  (6)  to  Exhibit  2  with  each  of  Hl,...,H15  inserted  in  the 
objective  produces  the  h*  values  reported  in  Table  1.  Every  one  of  the 
efficient  DMU's  has  received  a  rating  of  h*  =1  but  two  inefficient  DMU's— 
H10  and  HI 3— are  also  accorded  a  value  of  h*  =1  even  though  they  are 
inefficient.  The  six  DMU's  that  are  rated  as  inefficient,  with  h*  <1, 
are  accorded  these  values  by  comparison  with  certain  efficient  units  that 
comprise  an  efficiency  reference  set  for  the  inefficient  DMU  (see  Table  1). 
For  example,  H8  was  found  to  be  inefficient  by  direct  comparison  with  H4; 
and  HI 5  is  being  compared  directly  with  H4,  H6,  and  H7.  This  reference 
set,  we  need  only  note  here,  is  supplied  as  part  of  the  optimum  basis  in 
the  linear  programming  computations.  Hence  the  model  and  computing 
routines  supply  what  is  wanted  without  extra  effort  and,  furthermore, 
the  appearance  of  a  DMU  as  part  of  an  optimal  basis  ensures  that  it  is 
efficient  so  that  separate  computations  need  not  be  made  for  these 
entities  if  that  is  all  that  is  wanted. 

1/  Computer  codes  are  available  for  effecting  these  computations.  See  [  6  ]. 
New  software  by  I.  Ali  and  J.  Stutz  is  also  available  from  the  Center  for 
Cybernetic  Studies  at  The  University  of  Texas  at  Austin  which  detail  the 
efficient  facets  observed. 


I 

J 
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It  might  be  observed  that  the  two  inefficient  DMU's  that  were 
accorded  efficiency  values  of  h*  =1  have  no  such  reference  sets.  This 
suggests  that  they  have  special  properties  which  can  be  submitted  to 
further  analysis  by  means  of  the  non-Archmidean  formulations  that  we 
touched  on  earlier  in  the  textr^  We  shall  not  turn  aside  to  deal  with 
that  topic.  Instead  we  shall  simply  accept  this  identification  of  HI 0 
and  HI 3  as  a  possible  weakness  of  DEA  in  the  comparisons  we  are  making 
with  other  techniques  since  (as  in  this  case)  it  can  happen. 


1/Note  also  that  neither  HI 0  nor  HI 3  enter  into  the  reference  set  for  any 
other  DMU  . 


tiy 
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Efficient  DMU's 


Table  1 

DEA  Efficiency 
Rating  (E) _ 


Efficiency 
Reference  Set 


HI 

1.0 

H2 

1.0 

H3 

1.0 

H4 

1.0 

H5 

1.0 

H6 

1.0 

H7 

1.0 

Inefficient  DMU's 

DBA  Efficiency 
Rating  (E) 

Efficiency 
Reference  Set 

H8 

0.99 

H4 

H9 

0.98 

HI,  H2,  H6 

H10 

1,0 

Hll 

0.85 

H4,  H7 

H12 

0.99 

HI,  H4,  H6 

H13 

1.0 

H14 

0.99 

HI,  H4,  H6 

H15 

0.87 

H4,  H6 ,  H7 
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6.  Cost  Ratio  Analysis 

We  now  consider  how  a  manager,  e.  g.,  in  a  rate  setting  commission  for 
some  stated  might  determine  which  DMUs  are  more  and  less  efficient  when 
using  ratios,  a  widely  used  form  of  analysis  to  evaluate  financial  and 
operating  performance.  In  this  example,  all  the  inputs  are  jointly  used  by 
these  DMUs  to  produce  three  outputs  so  that  we  cannot  proceed  as  we  might 
in  the  single  output  case.  A  number  of  different  ratios  might  be  developed 
to  evaluate  different  sets  of  relationships  such  as  FTEs/patient, 

FTEs/severe  patient,  FTEs/regular  patient,  FTEs/teaching  output, 
bed  days/patient,  bed  days/severe  patient,  etc.  Such  a  set  of  ratios  does 
not  explicitly  recognize  the  joint  use  of  these  inputs  to  produce  these 
various  outputs.  In  addition,  for  the  set  of  ratios  calculated,  a  DMU  may 
be  among  the  highest  (least  efficient)  for  certain  ratios  and  lowest  (most 
efficient)  for  other  ratios.  This  leads  to  some  ambiguity  as  to  whether 
that  DMU  is  efficient  or  inefficient  and  calls  for  some  method  of  weighting 
or  ordering  the  importance  of  the  ratios  to  gain  some  overall  assessment  of 
efficiency  such  as  was  generated  using  DEA  in  Table  1. 

Rather  than  address  this  issue  directly,  we  will  focus  on  a  type  of 
unit  costing  ratio  analysis  that  is  often  applied  to  hospitals  and 
other  organizations  to  evaluate  DMU  performance.  By  design  we  can  say 
that  all  15  hospitals  (DWJs  )  paid  the  same  price  per  unit  for  each  type 

of  input  and  thus  ignore  possible  difficulties  which  arise  for  a  ratio 
analysis  when  this  is  not  the  case.  That  is,  we  can  combine  the  inputs  into 
dollar  units  without  the  confounding  effect  of  differing  input  costs.  Rather 

\J  For  instance,  see  [23]  and  [24]. 
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than  deal  with  all  these  outputs,  the  teaching  output  might  be  viewed  as  a 
by-product  or  secondary  output  and  the  patients  might  be  viewed  as  a  single 
output  rather  than  segregate  this  into  different  categories  of  severity. 

This  simplifying  procedure  is  not  wholly  defensible  from  a  cost  accounting 
standpoint.  Nevertheless,  in  the  absence  of  any  other  way  of  combining  and 
weighting  the  outputs,  similar  approaches  have  been  used  for  hospitals  as 
well  as  other  types  of  DMUs  (see  for  example  [23]),  and  this  is  the  way  we 
shall  proceed. 

Table  2  column  (A)  reflects  the  average  cost  per  patient  for  each  DMU. 

This  results  in  a  ranking  of  hospitals  reflected  by  the  parenthesized  number 

directly  to  the  riqht  of  the  average  cost  figure  in  Table  2.  The. lowest 
cost  (most  efficient)  DMU  is  ranked  1  and  highest  cost  (least  efficient)  DMU 
is  ranked  13.  This  ranking  erroneously  classifies  H13  (ranked  6)  as  more 
efficient  than  H3  (rank  7)  and  H6  (rank  9)  and  it  classifies  H9  as  more 
efficient  than  H6.  In  addition,  there  is  no  objective  means  for  determining 
the  cutoff  cost  level  to  segregate  efficient  and  inefficient  units. 

If  the  efficient  relative  costs  of  certain  outputs  are  known,  the  outputs 
can  be  weighted  to  reflect  a  coat  per  weighted  unit  of  output.  In  this  case  we 
know  the  efficient  cost  of  a  regular  patient  ($130)  and  a  severe  patient 
($170)  and  the  patient  units  can  therefore  be  weighted  to  value  each  severe 
patient  as  the  equivalent  of  170/130  ~  1.3  regular  patients.  For  example, 

HI  would  have  adjusted  patient  output  units  of  3000  regular  patients  + 

2000  x  1.3  severe  patients  for  an  adjusted  total  of  5600  patients.  Dividing 
this  patient  total  into  $775,500,  the  total  cost  for  HI  shown  in  Exhibit  3, 
results  in  $138.48,  the  case  mix  adjusted  average  cos^  shown  for  HI  in 
column  (B)  of  Table  2. 


Table  2 


1/ 


Single  Output  Measures 


Hospital 
Efficient  Units 

Average  Cost 
per  Patient 
(A) 

Case  Mix  Case  Mix  Adjusted  Average  Cost  per 

Adjusted  Patient  Segregated  into  High  and  Low 

Average  Cost  Levels  of  Teaching  Outputs 

per  Patient  Low*  High* 

(B)  (C)  (D) 

HI 

$155.10 

(2) 

$138. A8 

(A) 

$138.48 

12) 

H2 

163.32 

(5) 

138. A0 

(3) 

138.40 

<1, 

H3 

168.32 

(7) 

1A2.65 

(8) 

$142.65 

(3) 

HA 

160.10 

(A) 

1A2.9A 

(9) 

1A2.9A 

(6 ) 

H5 

158.38 

(3) 

137.73 

(2) 

137.73 

(2) 

H6 

170.15 

(9) 

1A0.12 

(5) 

140.12 

(3) 

H7 

1A2.60 

Cl) 

135.81 

(1) 

135.81 

(1) 

Inefficient  Units 

H8 

176.95 

(11) 

157.99 

(12)** 

157.99 

(6) 

H9 

168.32 

(7) 

1A2.6A 

( 7) 

142.64 

(5) 

H10 

169.69 

(8) 

161.61 

(1A)** 

161.61 

(7) 

Hll 

170.33 

(10) 

153.10 

(10) 

153.10 

(7) 

HI  2 

178.33 

(12) 

155.07 

(11) 

155.07 

(5) 

H13 

165.68 

(6) 

1A2.00 

(6) 

142.00 

(4) 

HI  A 

178.33 

(12) 

155.07 

(11) 

155.07 

(5)  . 

H15 

179. 7A 

(13) 

160. A8 

(13)** 

160.48 

(8) 

Mean 

167.02 

1A6.9A 

144.77 

149.42 

j 

f 

I 

Standard  Deviation 

8.82 

7.36 

9.66 

j 

] 

*  Low  teaching  outputs  were  50  units  and  high  teaching  outputs  were  100  units  as  per 
Exhibit  3,  Col.  6. 


••Hospitals  more  than  one  standard  deviation  over  average  cost 
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The  adjusted  cost  per  patient  is  reflected  in  column  (B)  of  Table  2 
with  the  new  ranking  in  parenthesis  immediately  to  the  right  of  the 
average  cost  per  day.  Even  with  this  (normally  not  available)  weighting 
of  patients  we  continue  to  have  a  misranking  with  inefficient  DMUs  H9 
and  H13  being  ranked  as  more  efficient  than  H3  and  H4.  If  we  further 
segregate  the  15  DMUs  by  the  third  output  (teaching),  as  is  sometimes 
done,  and  separate  them  based  on  those  with  high  (100  units)  versus  low 
(50  units)  teaching  outputs,  the  ranking  based  on  unit  costs  is  reflected 
in  columns  C  and  D  in  Table  2.  At  this  point,  we  have  achieved  an 
accurate  ranking  for  the  high  teaching  output  hospitals  but  we  still  have 
not  achieved  an  accurate  ranking  for  the  low  ones.  Because  we  have  only  two 
values  for  these  outputs,  at  50  and  100  "teaching  units,"  we  could  distinguish 
high  vs.  low  output  hospitals  fairly  easily  in  the  present  case,  but  generally 
there  will  be  many  more  values  to  consider  with  no  objective  guidance  available 
for  separating  high  from  low  teaching  output  values  and  the  difficulty  of 
distinguishing  efficient  from  inefficient  DMUs  will  then  be  compounded. 

The  problem  of  locating  a  point  beyond  which  DMUs  are  considered 
inefficient  is  typically  addressed  by  establishing  a  subjective  cutoff 
value*  even  though  there  is  no  assurance,  theoretical  or  otherwise,  that 
the  inefficient  units  will  be  accurately  located  through  this  process. 

For  example,  if  the  cutoff  was  set  at  one  standard  deviation  above  the 
mean  adjusted  cost  per  patient,  only  3  DMUs  (H8,  H10  and  HI 5)  would  be 
identified  as  inefficient  as  indicated  in  column  (B)  of  Table  2.- 

The  DEA  ratings  in  Table  1  do  not  lend  themselves  to  rankings  of 
the  kind  used  in  Table  2.  As  will  be  seen  below,  these  efficiency  measures 

1/At  0.674$a  =  5.95,  three  more  DMUs  (HI  1 ,  HI 2  and  H14)  would  be  added  to 
this  inefficient  set.  We  record  this  as  an  additional  possibility  for 
improving  this  kind  of  identification  even  though  most  of  the  commonly 
used  adjustments  are  in  the  direction  of  ko,  with  k  >  1. 
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are  intended  to  supply  estimates  of  excessive  resource  utilization  relative 
to  the  Efficiency  Reference  Sets  from  which  these  ratings  are  derived.  If, 
on  the  other  hand,  one  uses  the  estimated  resource  savings  as  a  basis  and 
accords  the  same  ranks  to  DMUs  with  equal  efficiency  ratings,  a  more 
informative  set  of  ranks  would  be  available  from  Table  1  than  Table  2.-^ 
Whether  ranked  or  not,  however.  Table  1  is  more  informative  than  Table  2 
provided,  of  course,  that  the  efficiency  values  exhibited  in  Table  1  are 
reasonably  accurate. 

7.  Regression  Analysis 

In  industries,  including  the  "health  industry,"  where  the  efficient 
input-output  technology  is  ambiguous  or  at  least  is  not  known  with  any 
real  precision,  regression  analysis  has  been  applied  in  order  to  gain 
"insights"  into  the  production  relationships  that  might  underlie  the 
observations  that  have  been  generated  from  past  utilization  of  these 
processes.  There  are,  of  course,  a  variety  of  problems  that  are 
encountered  when  using  traditional  regression  analyses  to  evaluate  the 
efficiency  of  individual  DMUs.  One  problem  in  most  such  studies  is 
that  one  relatively  smooth  relation  is  posited  to  obtain  the  parameter 
estimates  that  are  needed.  Another  problem  is  that  the  estimated 
parameter  values  are  based  on  least  squares  estimates  which 


1 /  ^  general  one  would  also  need  to  impute  dollar  magnitudes  or  other 
weights  to  the  potential  savings. 
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provide  "mean"  or  "central  tendency"  values  that  reflect  a  mixture  nf 
efficient  and  inefficient  behavior  in  the  data  set.-^  Thus,  even  if  the 
posited  functional  forms  are  correct,  the  estimated  regressions  will  only 
reflect  efficient  relationships  if  all  units  in  the  study  are  themselves 
efficient.  Whatever  reasons  may  be  used  to  justify  such  assumptions  in 
competitive  industries,  they  are  likely  to  be  much  weaker  in  not-for- 
profit  settings  such  as  education,  health,  and  government. 

Nevertheless  such  approaches  have  been  extensively  employed  and  so 
we  now  consider  the  extent  to  which  regression  analysis  as  it  has  been 
used,  e.  g.,  in  health  studies,  might  be  employed  to  identify  the 
inefficient  units  in  the  artificial  data  set.  In  the  process  we  shall 
also  locate  other  potential  problems  in  the  use  of  such  analyses  even 
when  we  can  validly  make  the  advantageous  assumptions  that 
all  DMUs  have  the  same  technology  and  pay  the  same  prices  for  all  inputs. 

One  part  of  our  analysis  involves  a  simple  linear  (additive)  regression 
model  in  which  total  cost  was  estimated  as  a  function  of  the  three  outputs 
produced  by  each  DMU.  The  results  were  as  follows: 

C  =  -95.300  +  152  y]  +  182.4  y2  +  1302  y3 
(8)  (22.2)  (767) 

where  C  =  Total  cost  per  year  (7) 

y.|  =  #  of  regular  patients  treated  per  year 
y2  *  #  of  severe  patients  treated  per  year 
y3  =  Training  units  provided  in  one  year 

1/  Recent  literature  has  begun  to  supply  a  variety  of  means  for  addressing 
some  of  these  problems  when  regression  estimates  for  securing  efficiency 
evaluations  are  wanted.  They  do  not  appear  to  be  very  satisfactory,  however, 
and  so  we  do  not  examine  them  here.  See  Banker,  Charnes,  Cooper  and 
Malndaratta  [  2  ].  We  confine  ourselves  only  to  those  types  of  regressions 
which  have  been  commonly  (and  widely)  employed.  See,  e.  g.,  [34). 
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The  standard  errors  noted  in  the  parentheses  below  each  coefficient 
indicate  high  levels  of  statistical  significance.  The  coefficient  signs 
are  positive,  as  required,  and  the  relation  between  the  y^  and  yg  (for 

regular  and  severe  patient)  coefficients  is  in  the  correct  (plausible) 

2 

direction.  A  high  R  value  of  0.97  suggests  a  good  fit  with  the 
observational  data  so,  by  standard  reasoning,  a  high  degree  of  cost 
variation  is  "explained"  by  these  independent  variables.-^ 

The  only  apparent  discrepancy  is  a  fixed  negative  cost  estimate  of 
$95,300.  This  value,  which  is  not  statistically  significant,  might  cause 
the  model  to  be  questioned  especially  in  cases  involving  hospitals  with 
relatively  small  outputs.  Hence  another  regression  with  its  total  cost 
intercept  fixed  at  zero  was  calculated.  We  do  not  reproduce  the  results 
here,  however,  since  (consistent  with  what  has  just  been  said)  the 
resulting  coefficient  values  did  not  differ  greatly  from  those  given  in  (7). 
Hence  the  latter  might  be  used  to  estimate  the  incremental  cost  per  unit 
of  each  output  as  in  the  second  column  of  the  following  tabulation: 


Output 

Estimated 

Incremental 

Cost 

Efficient 

Incremental 

Cost 

% 

Deviation 

yl 

$  152. 

$130 

17.0 

y2 

$  182.40 

$170 

7.3 

y3 

$1302. 

$500 

160.0 

I J  The  independent  variables  were  found  to  have  fairly  low  inter-correlations 
as  follows: 

Vz  ■  -°-371  r/iy3  *  -°-03i  Vs  •  •°-08- 
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Focusing  on  the  incremental  costs  in  this  manner  bypasses  the 

difficulties  associated  with  a  negative  intercept  value.  It  also 

corresponds  to  an  assumption  (not  often  stated  explicitly)  that  the  slope 

coefficients  may  still  parallel  the  true  incremental  efficiency  values, 

at  least  roughly,  in  a  manner  that  corresponds  to  a  shift  of  the  regression 

1  / 

plane  up  to  the  frontier  without  altering  its  slopes.-'  In  the  present 
case,  we  know  the  incremental  costs  for  efficient  operations  and  these  are 
supplied  in  the  third  column.  The  estimates  from  the  regression  are  high 
in  every  case.  Only  the  estimate  for  (=  severe  patients)  is  even 
tolerable  and  the  estimated  cost  for  y3  (=  teaching)  is  very  wide  of  the  mark. 

Another  use  of  such  regressions  is  to  evaluate  efficiencies  as  was  done 
by  Feldstein  [18]  in  his  now  classic  study  of  British  hospitals.  That  is 
the  actually  observed  outputs  for  each  of  HI  to  HI 5  would  be  inserted  in  an 
expression  like  (7)  and  the  resulting  total  cost  would  then  be  compared 
with  the  corresponding  actual  costs  at  this  hospital.-^  The  presence  of  a 
negative  intercept  value  could  be  troublesome,  however,  and  alternate 
forms  of  regression  functions  might  then  be  explored. 


1/This  method  of  parallel-shift  treatment  is  explicitly  incorporated  in 
some  of  the  "frontier  estimation"  methods  that  have  recently  been  devised. 
See  Forsund,  Lovell  and  Schmidt  [19]. 

2/ A  variety  of  adjustments  might  be  employed  to  allow  for  different  hospital 
characteristics  and  patient  mixes,  etc.  See  Feldstein  [18]  for  further 
discussion. 
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Another  type  of  function  that  has  been  coimonly  employed  in  hospital 
studies,  is  the  so-called  Cobb-Douglas  form.  This  form  has  the  advantage 
of  avoiding  the  possibility  of  negative  intercepts  and  since,  in  the 
present  data  set,  no  zero  outputs  are  present  for  any  of  the  hospitals  we 
can  also  avoid  difficulties  that  are  sometimes  experienced  from  this  quarter. 
Thus  we  now  turn  to  such  a  Cobb-Douglas  approach. 

In  logarithmic  form  our  estimated  relation  obtained  from  the  data  of 
Exhibit  3  is 


In  C  =  3.98  +  .62  In  y-j  +  .57  In  y ^  +  -10  In  y^  (8) 

(.04)  (.07)  (.05) 

which,  in  the  usual  Cobb-Douglas  representation,  becomes 


C  =  53.79  y 


0.62  „  0.57  „  0.10 


1 


(9) 


In  this  case  the  coefficients  in  (8)  and  hence  the  exponents  in  (9)  all 
appear  to  be  reasonable  as  well  as  significant.  In  sum,  however,  the 
exponent  values  (.62  +  .57  +  .20)  exceed  1  which,  being  significant, 
means  that  evidence  of  decreasing  returns  to  scale  is  Dresent,  or  at 
least  this  possibility  cannot  be  rejected.  In  our  case  this  may  reflect 


the  complementary  and  substitution  relations  that  are  known  to  be  present 

in  some  of  the  inputs. The  regression  does  not  detect  these  relations  in 

this  form,  however,  and  the  fact  that  it  results  in  a  significant  value 
2 

(with  R  =0.96)  could  lead  to  erroneous  recommendations  witn  respect  to 
decisions  on  the  scale  of  operations. 


1/  E.  g.,  as  reflected  in  A  x  =  y  when  going  from  x  =  Ay,  with  A  a  matrix 
of  positive  constants  as  in  (1).  Thus,  in  general,  A-1  will  have 
negative  as  well  as  positive  elements  reflecting  relations  of 
complementarity  as  well  as  substitution  among  the  various  inputs  used 
in  producing  these  output  combinations.  See  Sherman  [29]  for  further 
discussion. 
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If  we  now  consider  DMUs  as  potentially  inefficient  when  their  actual 
total  cost  exceed  the  estimated  total  cost  in  (9),  then  efficient  DMUs 
H2,  H6,  and  H7  would  be  erroneously  considered  inefficient  and  inefficient 
DMUs  Hll ,  HI 2 ,  H13,  and  HI 4  would  be  identified  as  efficient.  These  results 
together  with  the  results  of  our  preceding  analysis  are  drawn  together  and 
presented  in  Table  3.  In  identifying  which  DMUs  are  efficient  or  inefficient, 
DEA  has  evidently  done  better  than  the  others  with  the  exception  of  the  cost 
ratio  approach  when  the  latter  is  (a)  adjusted  for  case  mix  and/or  (b) 
identified  with  "low"  and  "high"  levels  of  teaching  outputs.  There  is,  of 
course,  a  degree  of  arbitrariness  present  in  these  cost  ratio  efficiency 
and  inefficiency  characterizations  that  provide  these  favorable  results  for 
comparison  with  DEA.  Furthermore  the  Case  Mix  adjustment  procedure  we 
used  presupposes  a  knowledge  of  the  efficient  cost  of  operations  and  this 
is  reflected  in  the  results  shown  in  both  columns  (B)  and  (C)  in  Table  3. 
Normally  these  costs  will  not  be  known  and  so  we  may  count  the  apparently 
favorable  results  of  these  ratio  analyses  as  proceeding  >om  an  assumed 
knowledge  that  will  generally  not  be  available.  This  snowledge  is  not 
required  by  DEA  and  hence  we  may  regard  it  as  being  superior  to  the  ratio 
analysis  in  these  respects  as  well  as  in  other-  respects  that  we  shall  begin 
to  examine  after  first  summarizing  some  of  our  other  findings  to  this  point 
as  follows: 

1.  Ratio  (cost)  analysis  and  regression  analysis  required  an 
arbitrary  rule  to  determine  which  DMUs  would  be  designated  as 
inefficient.  With  ratio  analysis,  the  mean  might  well  have  been 
lower  or  higher  depending  on  whether  there  were  more  or  fewer 
efficient  units  in  the  data  set.  Similarly,  regression  analysis 
might  also  have  a  lower  or  higher  cost  curve  depending  on  the 
relative  number  of  inefficient  units. 

2.  Ratio  analysis,  as  did  regression  analysis,  required  price  data 
and  other  adjustments  to  address  the  multiple  output  and  input 
situation  while  DEA  could  address  this  situation  directly.  In 
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addition,  the  ratios  would  be  confounded  if  DMUs  paid  different 
prices  for  similar  inputs.  For  example,  a  DMU  that  had  very  low 
prices  might  have  a  lower  average  cost  that  could  obscure  the 
presence  of  technical  (production)  inefficiencies.  Regression 
analysis  also  assumed  DMUs  had  the  same  costs/input,  and  different 
unit  input  costs  would  have  shifted  the  cost  function  and  could 
thereby  also  conceal  inefficiencies. 

3.  Regression  analysis  results  depended  on  the  selection  of  an 

appropriate  model  or  set  of  cost  relationships  and  nothing  in  the 
data  set  suggested  that  either  of  the  choices  were  not  appropriate. 
DEA,  however,  required  no  such  assumptions. 


There  are  other  points  that  can  also  be  made  as  we  move  beyond  mere 
classification  into  identifying  the  particular  inputs  where  inefficiencies 
occur  and  estimating  their  amounts.  This  will  be  dealt  with  in  the  sections 
that  follow. 
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Table  3 

Comparison  of  DEA,  ratio  analysis,  and  linear  regression  approaches 
ability  to  locate  Inefficient  DMU's 

E  =  DMU  rated  as  efficient 
I  =  DMU  rated  as  inefficient 
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(A) 


DEA  (1) 

Efficient  DMU's  Results 


(B)  (C) 

Case  Mix  Adjusted 


HI  E 

H2  E 

H3  E 

H4  E 

H5  E 

H6  E 

H7  E 

Inefficient  DMU’s 

H8  I 

H9  I 

H10  E 

HI  1  I 

H12  I 

Hi  3  E 

H14  I 

HI  5  I 

(1)  From  table  1 


E 

E 

E 

E 

E 

E 

E 

I 

E 

I 

E 

E 

E 

E 

I 


E 

E 

E 

E 

E 

E 

E 

I 

E 

I 

I 

I 

E 

I 

I 


(D) 


Av6r3op 

Jat|°  (2)  Cost/Patient  (3)  f^ression  (4) 
Analysis _  (Cobb/Douglas) 


E 

I 

E 

E 

E 

I 

I 

I 

I 

I 

E 

E 

E 

E 

I 


(2)  From  table  2  column  B  -  DMUs  with  cost/patient  greater  than  one  standard 
deviation  above  the  mean  used  to  identify  inefficient  DMUs. 

(3)  From  Table  2  columns  C  and  D  with  cost/patient  greater  than  one  standard 
deviation  above  the  mean  used  to  identify  inefficient  DMUs. 

(4)  Based  on  rule  that  DMUs  with  actual  total  cost  greater  than  estimated  total 
cost  (based  on  the  regression  model)  are  inefficient. 
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8.  Extensions 

Perhaps  the  easiest  approach  to  the  topic  of  identifying  the  sources 
and  estimating  the  amounts  of  inefficiency  present  in  each  DMU  is  to 
begin  with  a  specific  example.  We  therefore  begin  with  H15  as  an 
illustration  of  these  kinds  of  additional  uses  of  DEA.  This  hospital,  which 
is  inefficient,  has  already  been  discussed  in  association  with  HI  in 
Exhibit  3.  We  now  approach  it  in  a  different  manner  as  follows. 

★ 

First  consider  the  value  of  hQ  =  0.87  in  Table  1.  Here  we  shall  use 

this  value  to  obtain  the  results  shown  in  the  column  labelled  "Intensity  Adjusted 

Value"  in  Table  4.  Because  slack  values  also  need  to  be  considered  in 

assessing  efficiency  we  may  refer  to  these  h*  values  as  "intensity  factors" 

★ 

and  use  them  in  the  manner  of  the  tv  =  0.87  value  that  is  applied  to  each 

o 

of  the  inputs  in  Table  4.  The  value  which  is  then  obtained  in  the  case  of 
HI 5  can  then  be  compared  with  the  corresponding  value  shown  under  the 
column  labelled  "True  Efficiency  Value".  The  latter  are  the  values  of  the 
inputs  actually  needed  for  the  outputs  of  HIE  with  efficient  operations,  as 
obtained  from  the  efficient  coefficient  values  provided  in  Exhibit  1.  The 
maximum  discrepancy  of  $(139,200-130,000)=  $9,200  or,  approximately,  7 % 
occurs  in  the  case  of  Supply  $.  The  other  DEA  estimates  resulting  from 
the  intensity  adjustment  factor  applied  to  the  observed  inputs  are  within 
2%  and  0.3%,  respectively,  of  the  true  efficiency  values. 
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TABLE  4 

lllb  INTENSITY  ADJUSTMENT  AND  EFFICIENCY  VALUE 


Adjusted  Input  Values 


Efficient  Input  Values 


Observed  Intensity  Intensity 


Adjustments 


True 


Input 

Value 

Adjustment 

Factor 

Adjusted 

Value 

1 

Regular 

Severe 

Teach  Efficiency 
Units  Value 

FTE: 

26.5 

X 

0.87  = 

23.055 

.004  x  3,000  + 

.005  x  2,000 

+.03  x  50  =  23.5 

Bed  Days: 

47,370 

X 

0.87  = 

41,211.9 

(7 

x  3,000  + 

9  x  2,000) 

:  0.95*  =  41,052 

SUPPLY  $: 

160,000 

X 

0.87  = 

139,200 

20 

x  3,000  + 

30  x  2,000  + 

200  x  50  =130,000 

*0.95  =  vacancy  factor  for  efficient  production.  See  assumption 
(a)  in  Exhibit  1 . 


Evidently  our  h*  value  has  operational  significance  in  that  it 
indicates  "amounts"  of  inefficiency  that  are  present.  It  thus  differs 
from  the  index  numbers  and  like  approaches  that  are  sometimes  used  for 
efficiency  ratings.  See,  e.  g.,  the  index  constructed  by  Feldstein  [18] 
for  use  in  the  case  of  British  hospitals. 

As  indicated  earlier,  the  presence  of  slack  in  an  optimal  tableau  is 
also  to  be  considered  a  source  of  inefficiency,  and  these  data,  too, 
are  available  from  the  simplex  tableaus.  In  particular,  the  slack  value 
for  Supplies  in  the  optimal  solution  amounts  to  $11,880  and  955  Bed  Days 
of  slack  are  also  present.  When  these  amounts  are  subtracted  from  the 
Intensity  Adjusted  Values  in  rows  3  and  2  of  Table  4  new  estimates  for 

efficient  inputs  in  these  factors  become  $127,313  and  40,257  BD,  respectively. 
This  greatly  improves  the  efficiency  estimate  of  the  former 
along  with  some  worsening  of  the  latter.  All  estimates  are  now  within  about 
2%  of  the  true  efficiency  value. 
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It  is  not  contended  that  DEA  efficiency  estimates  will  always  be  this 
close  and,  indeed,  reference  to  Table  5  will  show  estimates  that  are  very 
wide  of  the  mark  for  H10  in  at  least  2  of  the  3  pertinent  input  categories. 

On  the  other  hand,  even  in  this  case  the  estimates  are  both  better  and  more 
detailed  than  those  obtained  from  the  ratio  and  regression  approaches 
discussed  earlier  in  this  article.  Also,  as  was  observed  in  our  discussion 
of  Table  1,  there  are  strong  reasons  to  suspect  the  hQ*  =  1  intensity 
values  for  H10  and  HI 3.  Elimination  of  these  two  hospitals  still  leaves 
HI  1  with  errors  in  the  range  of  10-lb*  for  three  of  the  input  estimates,  while  all 
of  the  other  errors  are  in  a  range  of  about  at  2%  or  less.  Furthermore 
this  record  is  considerably  improved  when  the  efficient  hospitals,  HI  to  H 7, 
are  added  to  the  list  since  in  their  case  the  estimates  all  have  zero  errors. 

This  seems  to  be  a  very  creditable  performance,  at  least  compared  to 
what  the  other  approaches  appear  to  offer  for  use  on  the  data  base  we 
have  erected.  Further  testing  will  also  be  required  both  on  other  data  bases 
and  in  actual  uses,  of  course,  and  improvements  in  the  methodology  and 
alternate  modeling  approaches  and  estimation  methods  will  also  need  to  be 
explored. 

Methods  by  which  such  testing  might  be  done  will  be  discussed  in  the 

next  section.  We  can  then  conclude  this  section  by  noting  that  still  other 

uses  of  DEA  are  also  possible.  For  instance,  what  we  have  been  doing  in 

this  section  amounts  to  projecting  each  DMU  onto  the  relevant  position  of 

the  efficiency  surface  in  conformance  with  the  methods  prescribed  in  [13]. 

Further  tradeoffs  may  then  be  effected  by  reference  to  the  marginal  rates 

★  * 

of  transformation  and/or  substitution  via  the  optimal  ur  and  v^  values 


which  may  be  secured  from  the  simplex  tableaus.  See  (6).  These  values  can 
provide  guidance  for  augmenting  or  contracting  the  inputs  and  outputs  of 
the  corresponding  DMU  and,  at  the  same  time,  provide  controls  and  guidance 

on  efficient  uses  by  the  managers  of  these  DMUs. 

★  * 

These  ur  and  v^  values  will  represent  estimates  which,  of  course,  may 
not  be  wholly  accurate.  The  same  is  true  of  the  similar  uses  of  regression 
estimates  but,  in  addition,  such  regression  estimates  can  be  expected  to  be 
very  wide  of  the  efficiency  values--as  should  be  clear  from  our  earlier 
discussions.  Indeed,  as  noted  in  [2],  the  estimates  of  such  substitution 
and  transformation  rates  generally  continue  to  be  very  far  from  the  true 
efficiency  values  even  when  the  simple  forms  of  regression  functions  used 
in  the  present  article  are  replaced  by  more  general  and  flexible  forms  and 
when  the  statistical  methods  used  are  specifically  directed  toward  frontier 
efficiency  estimates. 
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TABLE  5 

ESTIMATED  AND  TRUE  EFFICIENCY  VALUES 


H8  to  H  15 


HOSPITAL 

INPUTS 

OBSERVED 

VALUE 

"  Intensity 

ADJUSTED 

VALUE 

SLACK 

ESTIMATED 

EFFIC. 

VALUE 

TRUE 

EFFIC. 

VALUE 

ER 

FTE 

25.0 

24.75 

24.75 

25.0 

1.0 

H8 

BD 

49,475 

48,980 

8,425 

40,555 

1.2 

$S 

140,000 

138,600 

— 

138.600 

iSFriwitSTSfli 

1.0 

FTE 

24.5 

24.01 

24.01 

24.5 

H9 

43,160 

42,297 

«•« 

42,297 

43,158 

JS... 

161,700 

25,000 

136,700 

140.000 

RE 

77.0 

77.0 

77.0 

53.0 

45.0 

H10 

BD 

92,630 

92,630 

-- 

92,630 

92,632 

0.0 

$s 

340,000 

340,000 

— 

340,000 

280,000 

21.4 

FTE 

44.5 

37.8 

5.1 

32.7 

36.5 

10.4 

BD 

65,260 

55,471 

-  - 

55,471 

65,263 

15.0 

ss 

265,000 

225,250 

45,711 

179,539 

10.2 

m 

30.0 

29.7 

29.7 

30.0 

1.0 

60,000 

59,400 

9,476 

49.924 

1.2 

Wm 

170,000 

168,300 

— 

■T£)CT!i«H 

•mW 

1.0 

FTE 

43.5 

43.5 

43.5 

43.5 

HI  3 

BD 

81,110 

81,110 

81,1 1  u 

76.842 

5.6 

$s 

245,000 

245,000 

-- 

245,000 

2.1 

FTE 

30.0 

29.7 

29.7 

30.0 

1.0 

H14 

ggVnjWlijHil 

9,476 

49,924 

50,526 

1.2 

$S 

168,300 

— 

168  ..300 

1.0 

FTE 

26.5 

23.06 

23.06 

1.9 

HI  5 

BD 

47,370 

41,212 

955 

40,256 

1.9 

.  SL__ 

160,000 

139,200 

11,887 

127,313 

KHovlsMI 

2.1 

Note:  H10  and  HI 3  have  intensity  values  of  hQ*  =  1, 
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9.  Conclusion 


The  really  surprising  result  is  not  how  well  DEA  performed  on  our 

manufactured  data  base,  but  rather  the  poor  performance  of  the  econometric- 

statistical  models  we  employed.  These  models  are  representative  of  many 

analyses  that  have  been  employed  in  studies  used  to  draw  important  policy 

conclusions.  Two  recent  multi-million  dollar  studies  of  this  kind  that 

resulted  in  multi -volume  reports  with  important  findings  for  policy 

formation  are:  (1)  U.S.  Department  of  Health,  Education  and  Welfare, 

PSRO:  An  Initial  Evaluation  of  the  Professional  Standards  Review 

Organization  [in  Health  Care  Delivery]-^  and  (2)  U.S.  Office  of  Education, 

The  Follow  Through  Planned  Variation  Experiment  [for  Education  of 

2/ 

Disadvantaged  Children]. - 

The  questions  raised  by  our  across-DMU  regression  results  would  seem 
to  apply  a  fortiori  to  studies  like  these  since  in  our  case  the  design  of 
the  data  base  was  favorable  to  assumptions  such  as  a  common  technology  and 
a  common  price  structure  across  the  DMUs.  Assumptions  like  these  are  much 
less  likely  to  be  valid  for  regressions  used  in  applied  studies,  such  as  the 
kinds  we  just  cited. 

It  might  be  argued  that  it  is  unfair  to  level  criticisms  such  as  these 
at  regression  models  designed  to  handle  only  one  dependent  variable  at  a 

time  and  using  methods  of  estimation  directed  toward  average  rather  than 
3/ 

efficient  behavior.-  In  the  study  [2],  which  we  conducted  with  R.  Banker  and 

A.  Maindiratta,  however,  both  of  these  qualifications  were  accommodated. 

1/  See  [32].  See  also  [17]  for  further  discussion  and  suggestions  for 
alternative  approaches. 

2]  See  [33  ].  See  also  [12]  for  further  discussion  and  suggested  alternative 
approaches. 

3/  Note,  however,  the  study  by  Feldstein  [18]  which  was  conducted  in  just 
this  manner  and  numerous  other  studies  of  this  type  can  also  be  cited. 

See  also  the  study  by  Banker,  Conrad  and  Strauss  [  4]  which  consisted  of 
a  DEA  redo  of  a  previously  conducted  econometric  study  of  North  Carolina 
hospitals  (using  a  translog  function)  and  arrived  at  drastically  different 
conclusions  on  the  presence  of  returns  to  scale,  etc.,  which  had  been 
found  not  to  be  present  in  the  original  (econometric)  study. 
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In  that  study,  conducted  in  the  same  spirit  as  the  one  we  are  presently 
summarizing,  a  piecewise  Cobb-Douglas  function  with  one  output  as  the  dependent 
variable  was  used  to  represent  a  continuous  technology  with  increasing  and 
decreasing  returns  to  scale  in  its  various  segments.  Technical  as  well  as 
scale  inefficiencies  were  then  introduced  into  randomly  generated  observations 
as  a  basis  for  comparing  DEA  with  so-called  flexible  functional  form 
approaches  using  translog  regressions.  DEA  again  performed  very  well  but, 
perhaps  even  more  importantly,  the  statistical -econometric  approaches 
performed  poorly— not  only  relative  to  DEA  but  also  in  a  manner  that  was 
unsatisfactory  ger  se—in  both  technical  and  scale  efficiency  identification 
and  estimation.  Moreover,  the  estimation  methods  employed  for  the  regressions 
in  this  case  were  of  the  so-called  "corrected  least  squares"  varieties,  as 
specifically  designed  for  the  purpose  of  locating  and  estimating  efficiency 
frontiers.  See  [26]  and  [19]. 

One  possible  source  of  trouble,  we  think,  lies  not  merely  in  the 
estimation  methods  but  rather  in  an  approach— the  one  that  is  conrionly  taught 
and  employed— which  tries  to  capture  a  great  variety  of  behaviors  in  only 
relatively  smooth  and  simple  (e.  g.,  unconstrained)  functional  forms. 

Attempts  to  meet  these  difficulties  by  weighted  regressions,  outlier  analyses 
and  similar  approaches  do  not  really  deal  with  the  problem  in  a  sufficiently 
fundamental  way,  we  think,  and  other  alternatives  need  to  begin  to  be 
considered. 

The  optimizations  involved  in  these  DEA  and  statistical  approaches 
also  need  to  be  considered.  Generally  speaking  the  commonly  employed 
statistical  approaches  optimize  over  all  observations  while  DEA  optimizes 
relative  to  each.  Another  way  of  stating  this  is  to  note  that  a  complete 
DEA  analysis  will,  in  general,  involve  n  optimizations,  one  for  each 


observation,  while  the  usual  statistical  approach  involves  only  one. 

This  implies  that  differences  in  testing  for  results  and  checking 
for  possible  inferences  must  also  be  expected.  Because  it  is  directed 
toward  individual  observations,  DEA  is  also  directed  to  each  DMU  in  a 
way  which  suggests  this  as  a  fundamental  unit  of  test.  That  is,  the 
inferences  that  are  made  about  at  least  some  of  these  DMUs  can  and  should 
be  tested  by  on-site  observations  in  ways, and  with  results,  that  differ 
from  testing  statistical  estimates  for  general  types  of  class  properties 
effected  across  all  observations. 

Emphasis  on  individual  observations  is  one  way  to  distinguish  DEA 
from  customary  types  of  analyses.  But  DEA  is  directed  to  relative  efficiencies 
so  that  comparative  analyses  of  subsets  of  the  observations  are  also  in  order. 

Here,  too,  however,  differences  from  customary  approaches  need  to  be  noted. 

Consider,  for  instance,  the  estimates  of  marginal  rates  of  transformation 
and/or  substitution  that  are  associated  with  each  (efficient)  reference 
surface.-^  As  developed  in  [13], these  values  and  the  related  (efficient) 
marginal  rates  of  substitution  are  available  from  the  dual  variable  values 
which  are  to  be  found  in  the  optimal  simplex  tableaus. 

All  DMUs  which  have  the  same  efficient  reference  facet  will  have  the  same  dual 
variable  (  =  marginal  rate  of  transformation)-^  values.  What  this  means  can 
be  highlighted  by  contrasting  these  estimated  values  with  those  which  are 
available  from  ordinary  statistical  regression  approaches  applied  to 
individual  DMUs.  The  latter  can  also  be  used  to  obtain  estimates  of  such 
marginal  rates  of  transformation,  but  these  values  refer  only  to  the  behavior 

1/  In  the  case  of  single  outputs  these  would  be  identified  as  isoquants  and 
the  marginal  rates  of  transformation  would  become  the  marginal  productivities 
and  the  marginal  rates  of  substitution  that  could  then  also  be  derived 
from  these  estimates.  See  [13]. 

2/  See  also  [25]. 

3 /  As  explained  in  [11],  these  are  better  named  "virtual  transformation  rates." 
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of  each  such  DMU  whereas  the  DEA  values  are  obtained  from  the  efficient 
reference  set.  These  values  provide  the  efficient  marginal  rates  of  substi¬ 
tution  and  transformation  for  evaluating  the  effects  of  possible  resource 
augmentations  and  reductions.  The  estimates  that  are  thus  obtained  from 
DEA  indicate  what  will  (or  should)  be  done  with  efficient  usages  of  these 
augmentations  or  reductions— and  this  may  differ  greatly  from  the  past 
behavior  of  inefficient  DMUs. 

As  noted  earlier  in  this  article,  computer  codes  that  are  already 
available  will  provide  a  printout  that  identifies  the  efficient  DMUs  for 
each  relevant  facet.  The  means  for  effecting  a  comparative  analysis  is 
then  conveniently  at  hand.  Convenient  ways  for  assessing  the  sensitivity  of 
these  efficient  DMUs  to  data  variations  have  also  now  been  developed. 

See  [10].  A  way  is  thereby  opened  for  both  immediate  use  and  further 
research  on  DEA  approaches  to  areas  such  as,  e.  g.,  the  behavior  of  DMUs  in 
the  not-for-profit  sector  that  have  proved  so  resistant  to  other  types  of 
approaches. 

Having  identified  these  differences  and  their  possible  separate 
avenues  of  application,  testing  and  research,  we  can  probably  best  close 
on  a  somewhat  different  note  by  indicating  ways  in  which  the  two  approaches 
might  be  joined  together.  One  possibility  is  to  use  each  approach, 
regression  or  ratio  analysis  and  DEA,  to  check  on  or  fortify  the  other. — 

Other  possibilities  exist,  however,  which  might  briefly  be  sketched  as  follows. 

Aigner  and  Chu  in  [1],  essayed  a  new  approach  to  frontier  estimation 
by  means  of  what  would  now  be  called  "goal  programming with  only  one¬ 
sided  deviations  permitted  so  that,  in  general,  the  estimated  production 

Tj  See  [12]  for  further  discussion  on  different  conditions  which  might  lead 
to  one  approach  or  the  other  in  complementary  fashion  for  policy  guidance 
purposes . 

2/  This  was  originally  referred  to  as  "inequality  constrained  regressions." 

See  [10]  and  [8].  Although  not  available  at  the  time  of  the  Algner-Chu 
work  [1]  we  would  now  add  the  further  possibilities  that  are  now  available 
from  the  goal  interval  programming  approaches  described  in  [9]. 


function  (e.  g.,  a  Cobb-Douglas  form)  would  lie  on  or  above  all  of  the 
observed  output  values.  Confining  all  deviations  to  one  side  clearly  does 
not  exhaust  the  possibilities,  however,  and  one  may  go  on  to  prescribing 
proportions  of  the  total  deviations  or  even  deviations  for  individual 
observations  that  must  lie  on  one  side  or  the  other  of  an  estimated  frontier. 

In  a  similar  spirit,  C.  P.  Timmer  in  [30]  used  "chance  constrained 
programming"  formulations  and  concepts  to  effect  efficiency  estimates. 

Instead  of  utilizing  the  power  of  chance  constrained  programming,  e.  g.  to 
deal  with  different  proportions  and  even  different  probability  distributions, 
constraint  by  constraint,  Timmer  proceeded  in  an  entirely  different  direction 
and  in  the  spirit  of  a  "global"  statistical  analysis  discarded  "outlier" 
observations  one  after  another  until  he  achieved  what  he  regarded  as  "stable" 
estimates.  Notice,  however,  that  this  procedure  is  one  which  obliterates  a 
great  deal  of  information.  In  particular,  in  pursuit  of  one  global  (overall) 
property,-^  it  discards  efficient  DKUs  without  even  bothering  to  investigate 
them  individually. 

The  approaches  by  Aigner  and  Chu  [1]  and  by  Timmer  [30]  that  we  have 
just  described  involve  a  use  of  inequality  constrained  optimizations,  to 
be  sure,  but  they  otherwise  proceeded  in  the  spirit  of  classical  statistical 
approaches.  Something  more  may  also  be  accomplished  along  these  latter 
lines.  For  instance,  one  might  use  a  discriminant-function  or  cluster- 
analytic  approach  to  locate  subsets  of  the  original  points  which  have 
different  properties.  Hopefully  this  could  include  clusters  or  discriminant 
subsets  of  efficient  and  inefficient  points.  Separate  regressions  fitted 
to  these  subsets  might  then  yield  improved  ways  of  identifying  inefficiencies 
and  estimating  their  amounts. 

1/  This  is  contrary  to  the  spirit  of  individual  observation 
investigation  that  we  urged,  above,  and  for  which  the  kind  of  stability 
analysis  provided  in  [10]  is  now  available. 
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We  have  not  investigated  the  latter  types  of  topics,  as  we  shall  do 
in  future  papers,  for  the  simple  reason  that  we  sought  to  adhere  as  closely 
as  possible  to  the  kinds  of  approaches  that  have  generally  been  used  in  the 
kinds  of  studies  we  have  been  considering.  Notice  that  a  use  of  the 
discriminant  and/or  cluster  analysis  approaches  we  have  just  described 
involves  an  estimation  of  more  than  one  regression  relation  and  more  than  one 

optimization.  The  other  approaches  of  global  programming  and  chance 
constrained  programming  varieties,  as  in  Aigner  and  Chu  [1]  and  Timmer  [30], 
involve  inequality  constrained  relations  of  a  kind  that  are  similar  to  the 
ones  used  in  DEA.  Thus,  we  conclude  that  there  are  additional  avenues  of 
possible  relations  between  DEA  and  these  other  approaches  that  also  invite 
exploration. 


Structural  Model  (Efficient  HogpltfiX  Op»ratlcm« 
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lll-Tol.ll  Cost  -  (  23.5  m)(UO,000/nf)  +  <41,050  Bod  dny*  *  $J0/bod  dsy)  ♦  $130,000  Supplies  -  $775,500 
III 5-Tiiml  Cost  -  (  20.5  m:)($lU,0U0/m')  >  (  * 7 ,  JOB  O i-d  .luys  *  ) 1 0/licU  tiny)  ♦  $100,000  Supplied  -  $1198,700 


Emnnle  of  Construction  of  Data  Base  for  Hospitals  HI  (efficient)  «nd  H15  (Inefficient' 
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